The invention relates to tracking multiple simultaneous touches adjacent a two-dimensional (2D) touch panel.
2D touch panels based on capacitive sensing are in widespread use in a variety of applications. Typically, the 2D touch panels are actuated by a finger, either in direct contact, or through proximity (i.e. without contact). Sometimes a stylus is used instead. For example, 2D touch panels are used as touchpads of laptop computers, in control panels of domestic appliances such as microwave ovens and cooktops, and as overlays to displays on hand held devices, such as mobile telephones. Many other applications are known in the art.
For some applications, a 2D touch panel is designed to be able to detect two or more touches simultaneously, this capability often being referred to as “multitouch” in the art. For example, as is well known, computers are conventionally controlled by a mouse, which combines two or three sensors, namely the tracking ball for the cursor motion and two buttons for selection of icons at the cursor position. A mouse thus combines cursor motion through movement of the mouse device, and two finger actions for actuating the left and right mouse buttons. In a laptop, the mouse functions are provided by a touchpad with adjacent buttons. A user moves the cursor through sliding one finger over the touchpad area, and selects icons and so forth by actuation of the two “mouse” buttons with his or her thumb, or one or two other fingers.
Another example of a device that requires multiple simultaneous finger inputs is a hand-held games console, where typically the left and right thumbs are used to control different functions of the device, or jointly to control the same function. Controllers for in-flight entertainment systems often have a similar mode of operation.
More recently 2D touch panels have been programmed to recognize multitouch gestures such as pinch of a thumb and forefinger. A more classical example would be a music keyboard where playing a chord could be referred to as a multitouch gesture using modern industry parlance.
Typically, touches adjacent a touch panel are obtained, i.e. polled or sampled, over a series of time-frames separated by a time interval which may be fixed or variable. At every time interval that the touch panel is sampled, a set of co-ordinates for each detected touch are obtained.
Prior art methods for detecting multiple touches simultaneously are for example discussed in U.S. Pat. No. 5,825,352 [1], U.S. Pat. No. 6,888,536 [2] or US 2006/0097991 [3]. These techniques are used for detecting and processing multiple simultaneous touches on a sensor array so that a first finger can be used to control a cursor (similar to a conventional touch pad on a laptop computer) and a second finger to provide actuations (similar to a conventional mechanical button provided adjacent to a touch pad) for example.
In addition to being able to detect two or more touches simultaneously in a single frame it is also necessary to track touches that are detected in one sample time-frame of a touch panel to the next sample time-frame. This could be used for providing gestures on a trackpad or moving a cursor across a display, either using an independent touch panel or a touch panel incorporated into the display.
A method for tracking multiple touches has been previously proposed in our unpublished U.S. patent application 60/949,376 [4]. An example method for tracking two touches adjacent a touch panel is described below. For example, two touches are sensed at two positions in a first time-frame (t1) on a touch panel and after a given time interval, two touches are sensed at two positions in a second time-frame (t2). To determine which touch at time t1 tracks to which touch at time t2, the path lengths of all of the possible paths between the touch locations are calculated. The total distance for each of the combinations is found by using the x- and y-coordinates of the four touch positions. The combination with the lowest distance value represents the smallest tracking distance for the two objects which is deemed to be the most likely combination of tracking for the two objects. However, when tracking objects adjacent a touch panel, a maximum allowable tracking distance over which one object can track from one position to another could be applied. For example, if a first touch is detected on the position sensor at one time interval and then a second touch is detected at the next time interval, but the distance between the two touches was above the predetermined maximum distance, then the second touch is treated as a new object and not the same object moving from the first touch position.
The tracking algorithm described above can be extended to higher numbers of touches, but the number of calculations that are required for ‘n’ touches is n! Therefore, it becomes computationally difficult to track the touches as the number of touches increase, especially if a microcontroller is to be used. The tracking algorithm will also break down when multiple touches are closer together than their movement distance from frame-to-frame, since the result will depend on the order in which the touch coordinates are processed.
A prior art method for tracking multiple objects adjacent a touch 2D proximity sensor array is discussed in U.S. Pat. No. 6,888,536 [2]. The current locations of the surface contacts are predicted along existing trajectories using path positions and velocities measured from previous images. The closest active path for each group of contacts is found and the distance along the active path is recorded. Each group of contacts is paired with its closest path, providing the distance between them is less than the tracking radius. Any group which is not paired with an active or recently deactivated path is allocated as a new path, representing touch of a new finger on the surface. Any active path which cannot be paired with a group is deactivated, representing lift-off from the surface. This method uses full scale numeric processing requiring a digital signal processor or microprocessor, and is beyond the capabilities of a microcontroller.
There is therefore a need for a reliable multitouch tracking algorithm that can be carried out on a microcontroller or other integrated circuit device with limited processing power and memory capacity.